Federated Multi-Armed Bandits

Federated multi-armed bandits (FMAB) is a new bandit paradigm that parallels the federated learning (FL) framework in supervised learning. It is inspired by practical applications in cognitive radio and recommender systems, and enjoys features that are analogous to FL. This paper proposes a general framework of FMAB and then studies two specific federated bandit models. We first study the approximate model where the heterogeneous local models are random realizations of the global model from an unknown distribution. This model introduces a new uncertainty of client sampling, as the global model may not be reliably learned even if the finite local models are perfectly known. Furthermore, this uncertainty cannot be quantified a priori without knowledge of the suboptimality gap. We solve the approximate model by proposing Federated Double UCB (Fed2-UCB), which constructs a novel "double UCB" principle accounting for uncertainties from both arm and client sampling. We show that gradually admitting new clients is critical in achieving an O(log(T)) regret while explicitly considering the communication cost. The exact model, where the global bandit model is the exact average of heterogeneous local models, is then studied as a special case. We show that, somewhat surprisingly, the order-optimal regret can be achieved independent of the number of clients with a careful choice of the update periodicity. Experiments using both synthetic and real-world datasets corroborate the theoretical analysis and demonstrate the effectiveness and efficiency of the proposed algorithms.Comment: AAAI 2021, Camera Ready. Code is available at: https://github.com/ShenGroup/FMA

Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game

Offline reinforcement learning (RL) aims at learning an optimal strategy using a pre-collected dataset without further interactions with the environment. While various algorithms have been proposed for offline RL in the previous literature, the minimax optimal performance has only been (nearly) achieved for tabular Markov decision processes (MDPs). In this paper, we focus on offline RL with linear function approximation and propose two new algorithms, SPEVI+ and SPMVI+, for single-agent MDPs and two-player zero-sum Markov games (MGs), respectively. The proposed algorithms feature carefully crafted data splitting mechanisms and novel variance-reduction pessimistic estimators. Theoretical analysis demonstrates that they are capable of matching the performance lower bounds up to logarithmic factors. As a byproduct, a new performance lower bound is established for MGs, which tightens the existing results. To the best of our knowledge, these are the first computationally efficient and nearly minimax optimal algorithms for offline single-agent MDPs and MGs with linear function approximation

OCCS Classification and Treatment Algorithm for Comminuted Mandibular Fractures Based on 109 Patients and 11 Years Experiences: A Retrospective Study

Comminuted mandibular fractures (CMFs) pose significant challenges to surgeons for their serious complications and poor outcomes. We aimed at proposing a classification with treatment algorithm of each category for CMFs. Patients with CMFs were retrospectively reviewed and classified into five categories: Type I: relatively good occlusion, no or slightly displaced fragments, no continuity destruction or bone defect; Type II: relatively good occlusion, damaged morphology, low comminution degree but intact continuity without bone defect; Type III: damaged morphology and higher comminution degree with intact continuity and relatively good occlusion; Type IV: high comminution, impaired continuity and poor occlusion without segmental bone defect; Type V: segmental bone defect. Conservative treatment, open reduction and internal fixation or microvascular osteocutaneous free flap transplantation was performed, accordingly. Demographics, perioperative data, complications and reasons for reoperations were recorded. The chi-square test was used for statistical analysis. In total, 109 patients were included in the study. After surgery, in the following group, 5 manifested infections, 1 manifested bone non-union, and 2 experienced reoperations, while in the unfollowing group, 10 manifested infections, 5 manifested bone non-union and 8 experienced reoperations. The OCCS classification and algorithm for CMFs achieve better outcomes and with lower complication rate

Enhanced light-field image resolution via MLA translation

This work describes a method that effectively improves the spatial resolution of light-field images without sacrificing angular resolution. The method involves translating the microlens array (MLA) linearly in both x- and y-directions in multiple steps to achieve 4 ×, 9 ×, 16 × and 25 × spatial resolution improvements. Its effectiveness was firstly validated through simulations with synthetic light-field images, demonstrating that distinct spatial resolution increments can be achieved by shifting the MLA. An MLA-translation light-field camera was built based on an industrial light-field camera, with which detailed experimental tests were carried out on a 1951 USAF resolution chart and a calibration plate. Qualitative and quantitative results prove that MLA translations can significantly improve measurement accuracy in x- and y- directions while preserving z-direction accuracy. Finally, the MLA-translation light-field camera was used to image a MEMS chip to demonstrate that finer structures of the chip can be acquired successfully.Published versionThis work was funded by National Natural Science Foundation of China (12172222) ; Aero Engine Corporation of China (HFZL2020 CXY014-2) ; Fundamental Research Funds for the Central Universities

Status of lead accumulation in agricultural soils across China (1979-2016)

The first national-scale assessment of lead (Pb) contamination in agricultural soils across China was conducted based on > 1900 articles published between 1979 and 2016. Pb concentrations, temporal and spatial variations, and influencing factors were analyzed. Children's blood lead levels (BLLs) were also estimated using the integrated exposure uptake biokinetic (IEUBK) model. Pb concentrations in different areas of China varied greatly, which was closely associated with the distribution of Pb-related industries, especially Pb-zinc mine smelling, non-ferrous polymetallic mine smelting, e-waste recycling, and leaded gasoline consumption. The year 2000 was a significant transition year for Pb concentrations, with a rapid increase pre-2000 and a subsequent slow upward trend. Pb concentrations were found to be strongly associated with indicators of economic and social development including gross domestic product (GDP), population size, and vehicle ownership. Leaded gasoline, coal combustion, and non-ferrous smelling were the main sources of atmospheric Pb during the different periods. Predicted BLLs were higher in South China than those in the north. This study details the overall Pb contamination status of agricultural soils in China, and thus provides insights for policymakers with respect to pollution prevention measures

Supplementary document for Spatial resolution enhancement with line scan light-field imaging - 6620727.pdf

Supplemental Document